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Abstract 

Weather related disruptions account for seventy 
percent of the delays in the National Airspace System 
(NAS). A key component in the weather plan of the 
Next Generation of Air Transportation System 
(NextGen) is to assimilate observed weather 
information and probabilistic forecasts into the 
decision process of flight crews and air traffic 
controllers. In this research we explore supporting 
flight crew weather decision making through the 
development of a flight deck predicted weather 
display system that utilizes weather predictions 
generated by ground-based radar. This system 
integrates and presents this weather information, 
together with in-flight trajectory modification tools, 
within a cockpit display of traffic information (CDTI) 
prototype, that the CDTI features 2D and perspective 
3D visualization models of weather. The weather 
forecast products that we implemented were the 
Corridor Integrated Weather System (CIWS) and the 
Convective Weather Avoidance Model (CWAM), 
both developed by MIT Lincoln Lab. We evaluated 
the use of CIWS and CWAM for flight deck weather 
avoidance in two part-task experiments. Experiment 
1 compared pilots’ en route weather avoidance 
performance in four weather information conditions 
that differed in the type and amount of predicted 
forecast (CIWS current weather only, CIWS current 
and historical weather, CIWS current and forecast 
weather, CIWS current and forecast weather and 
CWAM predictions). Experiment 2 compared the use 
of perspective 3D and 2 V 2 D presentations of weather 
for flight deck weather avoidance. Results showed 
that pilots could take advantage of longer range 
predicted weather forecasts in performing en route 
weather avoidance but more research will be needed 
to determine what combinations of information are 
optimal and how best to present them. 


Introduction 

Weather related interruptions account for 
seventy percent of the delays in the U.S. National 
Airspace System (NAS) [1]. To meet the objective of 
expanding the capacity of the U.S. NAS in the Next 
Generation Air Transportation System (NextGen), 
new tools and technologies must be developed to 
help mitigate weather impact. A key component in 
NextGen’ s weather plan is to assimilate observed 
weather information and probabilistic forecasts into 
the decision making process of flight crews and air 
traffic controllers [2]. Here we report on two types of 
development that support flight deck weather 
decision making: enhancing information sources and 
improving information presentation. First, we discuss 
three concepts for enhancing weather information 
available to the flight deck. Second, we describe a 
prototype interface that implemented these concepts 
and a part-task experiment that evaluated the utility 
of these concepts. Third, we turn attention to the 
information presentation aspect, describing a part- 
task experiment that compared 2ViT> and 3D 
presentations of predicted weather. Finally, we 
discuss the implications of the results for how to best 
support pilot weather decision making in NextGen 
environments. 

Concepts for Flight Deck Weather 
Information Enhancement 

Weather information available to the flight deck 
today has limited range due to the constraint of 
airborne weather sensing technologies. One way to 
overcome the range limitation is to provide the flight 
deck with weather information collected using 
ground-based sensing technologies like NEXRAD. 
This idea is in line with NextGen’s goal to provide 
NAS users with same-time access to a unified 
aviation weather picture through a proposed 
infrastructure called Common Support Services- 



Weather (CSS-Wx) [3]. Access to NEXRAD weather 
has been made available mainly to general aviation 
cockpits through commercially available systems 
(e.g., Fore Flight, NOAA aviation weather system, 
etc). In the present research we explore the utility of 
presenting ground-based weather products on the 
commercial transport category flight deck. Our 
approach is to present current and forecast weather 
information from the Corridor Integrated Weather 
System (CIWS) on a flight deck weather and traffic 
display [4]. Developed by MIT Fincoln Faboratory, 
with funding support from the FAA and NASA, 
CIWS integrates thunderstorm forecasting 
technology with data from national weather service 
radars in the U.S. and Canada, and is used by air 
traffic controllers (ATC) to aid in the analysis of 
airspace congestion as a result of convective weather, 
the largest cause of air traffic delays in the NAS. 
CIWS is now in use at eight en route centers in the 
northeast U.S., six major terminal control areas, and 
the Aviation Research System Command Center. 

In this research we also explore enhancing 
weather information by making predictions from the 
Convective Weather Avoidance Model (CWAM) 
available to the pilot [5]-[7]. CWAM outputs three- 
dimensional probabilistic weather avoidance fields, 
which identify regions of airspace that pilots are 
likely to avoid due to the presence of convective 
weather. The motivation for CWAM development 
was to provide controllers with predictions of how 
pilots are likely to deviate for weather given a certain 
weather pattern and intensity. Predictions of 
avoidance decisions by others pilots confronted with 
similar weather patterns provides a perspective 
different from avoidance decisions based solely upon 
a physical model of the weather hazard. Instead, it is 
grounded in how other pilots evaluated riskiness and 
benefits when confronted with these weather patterns, 
and thus takes into account not only the weather 
patterns per se but also pilots evaluations of these 
patterns, and their impacts on flight. 

A third concept of weather information 
enhancement that we explore arose in the process of 
our exploring the use of CIWS weather forecasts. As 
with any weather forecast products, CIWS forecasts 
are probabilistic with varied levels of accuracy in 
predicting various types of convective weather 
development. Specifically, the ability of CIWS to 
predict storm cell movements far exceeds its ability 
to predict convective initiation [8]. Recognizing that 


securing enough bandwidth to reliably transmit 
ground-based weather data to the flight deck will be 
challenging, we examined a third concept for 
enhancing weather information on the flight deck by 
supporting pilots in generating their own storm 
movement predictions using information already 
available to them from airborne radar. Our proposal 
is to collect historical weather information along the 
flight path and allow pilots to visualize this 
information in a way that will support them in 
generating their own predictions of future weather 
development. Certainly pilots are not meteorologists 
and will not likely be able to generate forecasts at an 
accuracy level on par with weather forecast products 
like CIWS. However, this solution requires only 
existing information to implement. If this enhanced 
presentation of existing information proves useful to 
the pilots, it could become an affordable solution to 
improving flight deck weather decision making. 

In the next section we describe how we 
implemented these three enhancements for weather 
information display on the flight deck. 

Flight Deck Predicted Weather 
Presentation and Decision Interface 

Cockpit Situation Display (CSD) and the Route 
Assessment Tool (RAT) 

The Cockpit Situation Display (CSD), an 
extension of a Cockpit Display of Traffic Information 
(CDTI), is an interactive display prototype that has 
been in development in the Flight Deck Display 
Research Faboratory at NASA Ames Research 
Center for over a decade (Figure 1). The CSD 
supports both traditional 2D and advanced 3D 
visualization models, and depicts the 4D 

interrelationship of traffic, terrain, and weather using 
a cylindrical volume metaphor and fast time 
extrapolations. Designed to provide the basis for 4D 
Trajectory-Based Operation (TBO), the CSD also 
includes the Route Assessment Tool (RAT) which is 
integrated with the aircraft’s Flight Management 
System (FMS), and allows for in-flight trajectory re- 
planning. A standard computer mouse is presently 
used to interact with the CSD prototype. 

The RAT provides the functionality to create 
and visualize in-flight route modifications, downlink 
proposed route modifications to Air Traffic Control 
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Figure 1. Screenshot of the CSD with 
CIWS and CWAM 

(ATC), receive route modifications from ATC, and 
execute modifications. The RAT supports the 
addition of waypoints at arbitrary latitudes- 
longitudes, and deletion of waypoints, through both 
clicking and dragging-and-dropping mouse 
operations. For each waypoint, pilots can also adjust 
an associated flight altitude and speed, thus enabling 
4D trajectory in-flight planning. 

Time , Hazard , and Altitude Control Sliders 

Three control sliders were implemented in the 
CSD for the viewing of current and forecast weather 
information in 2ViT> and 3D presentation (seen in the 
lower left corner of Figure 1, and close up in Figure 
2). A 2 V 2 D presentation is where weather data from 
one altitude slice at a time is viewed in 2D (shown in 
Figure 1). Users however can obtain a picture of the 
3D weather information by consecutively viewing 2D 
weather over a range of altitudes. The three sliders 
are also available for controlling the viewing of 
perspective weather presentation shown in Figure 3. 

The Time slider has a range of -120 minutes (the 
past) to +120 minutes (the future), with the 0-minute 
point (the present) in the middle. Sliding downward 
allows for the viewing of historical weather 120 
minutes into the past; sliding upward allows for the 
viewing of predicted weather 120 minutes into the 
future. When the slider position is moved away from 



t 00 min. ALL 20K FT 

d 00 nm 


Figure 2. Screenshot of Time, Hazard, and 
Altitude sliders 

the 0-minute point, a blue bead representing ownship 
position appears at a time-corresponding position 
along ownship’s trajectory (Figure 4). Therefore, 
when the pilot uses the mouse to drag the slider to 
select different time intervals (past or future), 
weather and ownship position will update 
accordingly to reflect what happened (if sliding 
downward into the past) or is projected to happen (if 
sliding upward into the future) at the specified time 
interval. 
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Figure 3. Screenshot of 3D CIWS 
Presentation 





Figure 4. Blue Bead Shows Ownship Position 
40 Minutes Ahead 

The Hazard slider has two different types of 
implementations. When used in conjunction with the 
viewing of CIWS weather (as shown in Figure 2), it 
supports the viewing of three hazard levels: green, 
yellow, and red. Setting slider position at a given 
level shows weather hazards at that level and above. 
For example, setting slider position at the green level 
would show weather at the green, yellow and red 
hazard levels; setting slider position at the red level 
would only show weather at the red hazard level. 
When the Hazard slider is used in conjunction with 
CWAM predictions (not shown in Figure 2), it 
supports the viewing of CWAM polygons at different 
probability levels, ranging from 20% to 90%. The 
percentages indicate the proportion of pilots 
predicted by CWAM to avoid the associated polygon 
region. In general, higher percentage values are 
associated with smaller polygons centering on highly 
hazardous weather (red) because these areas are 
typically avoided by a greater proportion of the pilots 
(Figure 5). 

The Altitude slider supports the viewing of 
CIWS and CWAM information at a given altitude. 
When used in conjunction with CIWS in perspective 
3D presentation, the altitude slider limits the viewing 
of 3D weather to the specified altitude and above. 
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Figure 5. CWAM Polygons at Four 
Probability Levels 


Experiment 1: Utility of Historical and 
Forecast Weather Information on 
Flight Deck Weather Decision Making 

The purpose of Experiment 1 was to evaluate 
pilot weather avoidance performance using three 
different types of weather information: CIWS, 
CWAM, and History. In a part-task setting, 
participants flew 48 flight scenarios that were 2 hours 
long in simulated time (but varying in real time 
duration) in the en route phase, and modified the 
trajectories when necessary to deviate for weather. 
To simplify matters, only a single aircraft (ownship) 
was present in the scenarios; traffic was not a 
consideration in this experiment. We evaluated 
objectively measured weather avoidance performance 
as well as subjective ratings of the various sources of 
weather information. 

Method 

Participants 

Sixteen transport pilots with high-altitude flight 
experience participated in the study and were 
compensated $25/hr. 

Apparatus 

The experiment was conducted using a personal 
computer (PC) equipped with a 30” LCD display. 
Pilots manipulated the CSD using a computer mouse. 







Design 

On each trial, pilots were presented with a 2- 
hour segment of an en route flight with one of four 
types of weather information: 

• Basic: Pilots only had access to CIWS current 
weather information updated every 2.5 
minutes. 

• History: In addition to CIWS current weather 
information and its updates, pilots had access 
to historical weather information from the past 
2 hours at 10-minute resolution. 

• CIWS: In addition to CIWS current weather 
information and its updates, pilots had access 
to 2-hour CIWS weather forecast information 
with forecast updated every 5 minutes. 

• CWAM: In the CWAM condition, in addition 
to CIWS current and forecast information and 
its updates, pilots had access to CWAM 
predictions. Because CWAM predictions are 
computed based on CIWS information, 
CWAM predictions were updated at the same 
cycles as CIWS information. 

Weather Scenarios 

Historical CIWS weather data obtained from the 
NASA NextGen ATM Data Warehouse provided the 
source of weather scenarios [9]. More than 50 
samples in 2-hour long segments were taken from 
July to August in 2011. Weather scenario samples 
included all information from CIWS product’s native 
update cycles (current weather updates every 2.5 
minutes and 2-hour forecast updates every 5 
minutes). Custom software was developed to convert 
raw CIWS data format (netCDF-4/HDF5) to formats 
usable by the CSD and to generate CWAM 
predictions. 

Once the weather samples were collected, the 
next step was to design 2-hour long flight trajectories 
that traversed a section of the weather impacted 
airspace. Because the 2-hour flight trajectory was 
meant to simulate a segment of a possible en route 
flight where, as part of an experimental constraint, 
pilots were not allowed to modify the end points, one 
criterion for designing the trajectories was that they 
began and ended with ownship flying clear of 
weather conflicts. 

One criteria used for the selection of the weather 
impacted airspaces used in this experiment was the 
associated accuracy of the weather forecasts for those 


airspaces. Attempts were made to select weather 
scenarios that varied in terms of forecast accuracy to 
reflect the probabilistic nature of forecasts in general. 
CIWS datasets include a Forecast Accuracy Score 
Product, which indexes accuracy by comparing 30, 
60, and 120 minute weather predictions with the 
actual weather. Scores come in 5% increments (up to 
100%) [10], [11], with higher percentage values 
corresponding to a greater match between forecast 
and current weather. Because forecast accuracy 
scores are only available for a 300x300NM area 
around each of the 77 defined home regions (mostly 
major airports in the U.S. and Canada) [12], flight 
trajectories were designed to fly through at least one 
home region so to make it possible to associate a 
forecast accuracy score with each of those 
trajectories. 

Among those weather scenarios collected, 48 
were used for experimental trials and 3 were used for 
practice. The 48 trials were divided into 4 groups of 
12 with roughly matching mean forecast accuracy 
(40% to 42%). For different pilots, the 4 groups of 
scenarios were paired up with different weather 
information conditions so that across participants 
different weather scenarios were used in all 4 weather 
information conditions. 

Procedure 

On each trial, pilots were presented with a 
unique weather scenario and a 2-hour section of 
flight plan in the en route phase. Their task was to 
guide their flight safely and efficiently through 
weather using the provided thunderstorm avoidance 
criteria: 

These are minimum standards: 

If Tops < FL 300 avoid thunderstorm by at least 5 

NM 

If Tops > FL 300 avoid thunderstorm by at least 20 

NM, clear tops by at least 5000’ 

Pilots had access to the applicable slider controls 
for manipulating the viewing of historical or forecast 
weather information, and the RAT for planning and 
executing path modifications. 

To accommodate 48 trials of 2-hour 
weather/flight scenarios within a reasonable amount 
of testing time, scenarios were fast-forwarded at non- 
critical periods of the flight. Specifically, each 2-hour 
scenario was divided into 6 20-minute segments. The 
beginning of each segment was a “real-time” period, 



during which pilots could evaluate ownship trajectory 
for weather impact and modify it if necessary. Once 
pilots made a modification, or decided that none was 
needed, they executed the modified or unmodified 
trajectory using RAT, and the scenario was 
automatically fast forwarded to the beginning of the 
next segment. During the fast-forward period, pilots 
could view the progression of the flight and the 
evolution of weather but could not make path 
modifications. Pilots were made aware of this 
procedure during training and asked to plan their 
paths to avoid weather accordingly. 

A total of 48 trials were divided into 4 blocks, 
one for each weather information condition. The 
order of the weather conditions was counterbalanced 
across participants. Pilots received the corresponding 
training for a particular weather condition right 
before that block of trials. The training involved 
verbal instructions and hands-on exercises, followed 
by self-paced practice runs. Pilots were asked to 
practice until they felt comfortable using the newly 
learned information source. After pilots completed all 
blocks of trials, they filled out an online 
questionnaire designed to solicit their subjective 
evaluations on various aspects of the scenario. 

On all of the trials, ownship was initialized with 
an altitude of 28000 feet and a ground speed of 464 
knots. No wind information was provided; pilots 
were instructed to infer wind direction based on the 
observed or forecasted movement of the storm cells. 

Only the Altitude slider, initially preset to 
25000, was activated and implemented in the same 
way in all four weather conditions. The Time slider 
was not activated in the Basic condition. It was 
activated for sliding between 0 and -120 minutes (the 
past) in the History condition, and between 0 and 120 
minutes (the future) in the CIWS and CWAM 
conditions. The Hazard slider showed green, yellow, 
and red hazard levels in the Basic, History and CIWS 
conditions, and was preset at the yellow level at the 
beginning of each trial. In the CWAM condition the 
Hazard slider was used for controlling the weather 
avoidance polygon probabilities instead of the three 
CIWS hazard levels. Because no slider control was 
available for the selective viewing of CIWS hazards 
in the CWAM condition weather from all hazard 
levels (green, yellow, and red) were continuingly 
shown. The initial CWAM percentage was set at the 
50% level. 


Results and Discussion 

Four of the sixteen pilots were excluded from 
most analyses: three due to failure to follow 
instruction to utilize historical weather information in 
the History condition, and one due to failure to 
follow instruction to minimize numbers of climbs. 1 
Results reported here are based on the remaining 
twelve pilots, who together fulfilled a complete 
counterbalancing of the four weather information 
conditions. 

Weather Avoidance Performance 

In our examination of weather avoidance, our 
measures only capture how the presentations affected 
strategic and not tactical guidance since pilots were 
only allowed to modify their path every 20 minutes in 
the simulated scenarios. Therefore, these results 
reflect the utility and acceptance of the various 
presentations for making strategic guidance decisions 
without the luxury of fine tuning their trajectories by 
requesting deviations as they approach weather 
systems. 

Weather avoidance performance was evaluated 
with regard to the safety and efficiency of flight 
paths. We used proximity to weather as the measure 
for safety. Specifically, for proximity we examined 1) 
the percentages of time flight paths came closer than 
20NM to the green and red hazard level regions of 
weather and, 2) the closest a flight came to each of 
these regions during a trial. We chose to focus on the 
green and red hazard levels because proximity to 
green measures proximity to weather of any intensity, 
while proximity to red signifies proximity to an 
extremely hazardous region. We examined coming 
within 20 NM because keeping 20NM from the green 
hazard level is often used as the guidance for weather 
avoidance as recommended by FAA’s Aeronautical 
Information Manual with regard to thunderstorm 
flying (although the actual recommended guidance 
given to pilots allowed for closer approaches if echo 
tops were below 30000 feet) [13]. 


1 We were informed by the early participants that the echo tops of 
some of the storms in our scenario were low enough that they 
could be bypassed altogether through a simple climb operation 
given the appropriate types of aircraft. As a result we instructed 
the participants to avoid climbs by having them assume the initial 
altitude was most efficient for their flight. 



Overall, pilots flew rather safely around 
weather, and penetration of the 20NM stand-off 
boundary rarely occurred (less than 10% and 5% 
penetration with respect to the green and red hazards, 
respectively). Table 1 summarizes the percentage of 
penetration and closest distance to weather by 
weather condition (shown as figures outside the 
parentheses). Results from a repeated measure 
within-subject Analysis of Variance (ANOVA) on 
the percentage of time penetrating the green hazard 
found a marginal main effect of weather information 
condition, F(3,33) = 2.83, p < .06. No difference was 
found for the percentage of time penetrating the red 
hazard level (p > .2). Echoing these results, results 
from an ANOVA on the closest distances to weather 
at the green hazard level found a significant main 
effect of weather information condition, F(3,33) = 
5.42, p < .01, while no effect of weather information 
condition was found for the closest distances to 
weather at the red hazard level (p > .2). As shown in 
Table 1, pilots flew closest to the green and red 
hazards when they had CIWS forecasts (mean closest 
distances 2.0 NM and 9.8 NM), and furthest from the 
weather when they had CIWS forecasts with CWAM 
(mean closest distances 7.4 NM and 12.0 NM). 

We also examined the closest proximity to 
weather with regard to CIWS forecast accuracy by 
doing a median split and dividing weather scenarios 
roughly into two halves based on their prediction 
accuracy scores. The results are summarized in the 
parentheses in Table 1 (based on scenarios with 
relatively low and high prediction accuracy, 
respectively). Note that the number of weather 
scenarios within each condition was already limited 
(12 each), thus results based on further reduced sets 
that contained as few as 4 scenarios should be 
interpreted with caution. Nonetheless, some patterns 
appear to emerge. First, pilots in general came closer 
to weather on trials with forecasts that had relatively 
lower prediction accuracy than on trials with 
forecasts that had relatively higher prediction 
accuracy. However, because the same pattern is 
observed in the Basic condition, where forecasts were 
not presented to the pilots, it is likely that the 
differences were due largely to a correlation between 
the type of convective activities and forecast 
accuracy (i.e., lower accuracy associated with 
prediction of growth and decay vs. higher accuracy 
associated with prediction of movement). Second, the 
closest distances to weather scenarios of similar 


Table 1. Experiment 1 Weather Avoidance 
Performance Results 


Weather 

Condition 

Hazard Level 

Green 

Red 

Basic 

% penetration 
closest distance 

8.5% (9.0%/8.0%) 

3.3% (3.8%/2.5%) 

4.8 (2. 7/6. 9) 

11.3 (9.3/13.4) 

History 
% penetration 
closest distance 

8.0% (8.0%/8.0%) 

3.3%(3.8%/2.5%) 

4.1 (2.2/6. 1) 

11.2(8.8/13.6) 

CIWS 

% penetration 
closest distance 

9.1%(8.0%/10.0%) 

3.9%(3.8/%/4.5%) 

2.0 (2.7/1. 2) 

9.8(8.6/10.9) 

CWAM 
% penetration 
closest distance 

6.6% (8.0%/5.0%) 

3.0% (3.6%/1.9%) 

7.4 (4.3/10.5) 

12.0(8.9/15.2) 


prediction accuracy were equivalent across the four 
conditions (except in the case of green hazard, 
CWAM condition). The differences in the closest 
distance to weather observed in the aggregated results 
were mostly due to scenarios with higher prediction 
accuracy. Understandably, pilots did not have 
information on accuracy scores and thus could not 
have approached weather differently according to 
how accurate the predictions were. It is likely though 
that pilots approached weather differently based on 
the predominant type of convective activities they 
perceived. 

Overall, the results of the weather proximity 
analysis showed that the predicted CIWS with 
CWAM condition resulted in pilots planning routes 
that held the greatest distance from the weather. 
Conversely, predicted CIWS without CWAM 
resulted in planned routes with least separation from 
the weather. The CWAM result was not unexpected 
since it presented a contour that surrounded the 
weather cells, providing effectively an additional 
buffer. Thus if pilots attempted to stay outside of the 
contour they would be even further from the weather. 
The performance with the predicted CIWS alone was 
less expected. One explanation for this may be 
overconfidence in the CIWS prediction, leading 
pilots to attempt to plan routes that skirt closer to the 
storms. 

Similarly, the results of path stretch, used as the 
measure for efficiency, showed a corresponding 
inverse relationship with those of proximity to 
weather. Path stretch was measured by the percentage 




of increase in length of the actual path flown relative 
to the planned path. Overall the actual paths flown 
were about 5% longer (5.6%, 5.2%, 5.0%, and 7.0% 
in the Basic, History, CIWS, CWAM conditions, 
respectively). Specifically, the results showed the 
greatest path stretch when pilots flew with CWAM 
predictions (7.0%) and the least when they flew with 
CIWS forecasts (5.0%), compared to the basic 
current weather only condition. Results from an 
ANOVA confirmed the significant effect of weather 
condition, F(3,33) = 3.20, p < .05. 


Subjective Evaluations 

Pilots rated the weather information in terms of 
utility in improving flight efficiency and safety, as 
well as reducing time on task and workload on a 5- 
point Likert scale, from strongly disagree (1) to 
strongly agree (5). They were instructed to rate the 
information relative to specific comparison 
conditions (History relative to Basic, CIWS relative 
to Basic, and CWAM relative to CIWS). Overall 
pilots showed relatively favorable opinions on these 
information sources (greater than 3.5 on average) 
(Table 2). 


Table 2. Experiment 1 Subjective Evaluation Results 


Weather 

Condition 

Areas 

Efficiency 

Safety 

Time 

Workload 

History 

4.17* (3.57*) 

4.17} (3.75*) 

3.50* (3.25) 

3.58 (3.44) 

CIWS 

4.50* ( 4.63 *) 

4.33* (4.50'*) 

3.83 (3.55*) 

4.00* (3.94*) 

CWAM 

3.75 (3.56) 

3.92* (3.69) 

3.92* (3.63) 

3.92* (3.63) 


Table 2 summarizes pilot ratings on weather 
information in areas of efficiency, safety, time, and 
workload. Means in parentheses were from all 16 
participants, including those who did not use history 
information. Means outside parentheses were from 12 
participants whose data were used for all other 
analyses. A rating of 3.0 signifies a neutral rating, 
meaning that the condition did not differ from the 
comparison condition on the dimension being rated. 
Therefore, significance ratings are assessed for 
difference between mean rating and 3.0. (fp < .05, Jp 
<. 01 ). 

Interpretation of the subjective ratings must take 
into account that four subjects were excluded from 
the analysis, in particular the three that were excluded 
because they did not utilize the historical weather 
information, even though they were trained in its use 
and instructed to use it. For the ratings data this 
exclusion introduces a potential bias, because some 
of these pilots found the weather history of 
insufficient value to use it. It is not clear if their 
evaluation of History would have risen (or fallen 
further) if they had used it, so mean ratings with and 
without these four participants’ data, are shown in 
Table 2. Table 2 also reports the results of t-tests of 
the significance of the difference between the mean 
ratings and a neutral value of 3.0. 


Relative to the Basic condition, pilots found it 
more useful to be given forecast information with 
CIWS. Seven of the eight tests comparing the means 
with the neutral rating (3.0) found statistical 
significance. 

Relative to the CIWS with forecast condition, 
pilots’ mean ratings indicate that they believed 
presentations adding CWAM weather avoidance 
fields were often better but not by much. Only three 
of the eight tests comparing the means with the 
neutral rating (3.0) found statistical significance. 

Although the History condition did not seem to 
affect safety and efficiency of flight based on the 
performance results, pilots who used it 
overwhelmingly found it useful to be able to 
visualize weather history relative to relying on their 
own memory (4.17). Tests for the History condition 
showed that participants’ ratings for weather history 
information were significantly above neutral (3.0) for 
efficiency and safety, even when the three excluded 
participants were added to the analysis. However, for 
time and workload only the rating for time with the 
four participants excluded was significantly greater 
than 3.0. 




Finally, in another question pilots were asked to 
indicate the extent of forecast range that they 
consider could be used effectively (Figure 6). Results 
showed that almost all of the pilots thought they 
could effectively use forecasts of up to an hour in the 
future. 


In summary, even without any weather 
information enhancement, as in the Basic condition, 
pilots flew the scenarios safely through weather with 
minimal penetration. However, weather information 
enhancements, particularly CIWS and CWAM, 
affected weather avoidance decisions in opposite 
ways. Specifically, the addition of CIWS forecasts 
resulted in more penetration into weather at the green 
hazard level and less path stretch. Conversely, the 
addition of CWAM predictions resulted in less 
penetration into weather at the green hazard level and 
more path stretch. Lastly, avoidance performance 
based on weather history in general did not differ 
from that based on current weather only. However, 
pilots found it useful to be able to visualize weather 
history. 


Extent of Forecast Range 
That Can Be Used Effectively 



Forecast Range 


Figure 6. Experiment 1 Subjective Effective 
Forecast Range 


Experiment 2: 2%D vs. Perspective 3D 
CIWS Presentations on Weather 
Decision Making 

Advances in computer graphics technologies 
combined with the increasing availability of 
affordable hardware solutions make it possible to 
render and present complex perspective 3D images 
for real time use. For phenomenon like weather 
which is highly spatial in nature, there is an intuitive 


appeal to present it in perspective 3D, rather than 2D, 
to get closer to how it appears in the physical world. 
However, research has shown that 2D and 
perspective 3D displays each has advantages and 
disadvantages depending on the tasks and situations 
[14]— [16]. Specifically, 3D presentation is better for 
shape and layout information whereas 2D 

presentation is better for precise orientation and 
positioning [16] and understanding the relative 
locations between objects [14]. 

The purpose of Experiment 2 was to evaluate the 
use of 2 V 2 D and perspective 3D presentations of 
weather for flight deck weather avoidance. In 
particular, we were interested in how these two 
presentation modes fare in a task like weather 
avoidance that demands both an understanding of the 
general layout of weather and precise positions 
between the trajectory of ownship and weather 
hazards. The task was identical to that of Experiment 
1, flying multiple 2-hour segments of trajectories in 
en route airspace safely through weather. There were 
two weather presentation conditions, 2!AD and 
perspective 3D. Sixteen participants flew 30 flight 
scenarios, 15 in each of the conditions. We evaluated 
the same objective and subjective measures as well as 
how much time pilots spent in 2D vs. 3D viewing (in 
perspective 3D condition). 

Method 

The method in Experiment 2 was identical to 
that of Experiment 1 with the following exceptions. 
First, there was a total of 30 trials divided into 2 
blocks, one for each weather presentation condition. 
These 30 trials featured new weather scenarios 
sampled from June to September, 2012. Each trial 
began with the display preset in the presentation 
mode being tested (top-down 2D view for 2 V 2 D and 
perspective view for perspective 3D). In the 3D 
condition pilots could freely rotate the display, 
turning it into 2D if so desired. The 2!AD presentation 
could not be rotated out of the 2D view. The hazard 
level was preset to include red, yellow, and green 
levels (i.e., ALL). The altitude of ownship was preset 
to 38000 feet and altitude changes in the planned 
trajectory were not discouraged as they were in 
Experiment 1. CIWS current and predicted weather 
were used as the source of weather information in 
both presentation conditions. 


Results and Discussion 

All analyses were based on all sixteen 
participants. 

Weather Avoidance Performance 

Overall, the two presentation conditions 
produced equivalent results in terms of avoidance 
performance (Table 3). Penetration within 20NM of 
the green and red hazard regions occurred 12.0% and 
8.2% of the time respectively with 2ViT> presentation 
and 12.0% and 7.8% of the time with perspective 3D 
presentation. The closest proximity to weather was 
also equivalent in the two conditions, around 1.1NM 
and 1.2NM for the green and 4.0NM and 4.5NM for 
the red hazard regions, respectively. Not surprisingly, 
the amount of path stretch was also equivalent in the 
two conditions, both around 4.2%. The two different 
presentations also did not result in any difference in 
the amount of time spent on weather evaluation, both 
around 33 sec. The two conditions however had a 
small difference on the average number of altitude 
changes per trial. The altitude of ownship changed 
1.5 times on average per trial in the 2ViT> condition 
and 1.3 times in the perspective 3D condition, 
F(l,15) = 6.05, p < .05. 

Subjective Evaluations 

Although there was no difference in weather 
avoidance performance between the two presentation 
conditions, pilots consistently preferred the 


Table 3. Experiment 2 Weather Avoidance 
Performance 


Presentation 

Hazard Level 

Condition 

Green 

Red 

2V 2 D 

% penetration 
closest distance 

12.0% (1 1.0%/12.7%) 

8.2% (7.1%/9.2%) 

1.1 (1. 2/0.9) 

4.0 (4.3/3. 8) 

3D 

% penetration 
closest distance 

12.0% (12.5%/1 1.7%) 

7 . 8 %(7 . 8 %/8 . 0%) 

1.2 (-0.2/2. 4) 

4.5 (3. 7/5.2) 


perspective 3D presentation to the 2ViT> presentation. 
In particular, they preferred 3D in terms of their 
ability to assess the height and spread of weather and 
the ability to plan alternative trajectories (Table 4). 
As a whole, pilots overwhelmingly preferred 
perspective 3D presentation over 2ViT> (80% vs. 
20%). Perspective 3D presentation was also 
considered significantly easier to use than 2ViT> 
presentation for assessing the height of weather (t = - 
4.53, p < .001), assessing the penetration of weather 
by ownship’s planned trajectory (t = -4.03, p < .005), 
assessing the penetration of weather by ownship’s 
trajectory being planned while using RAT (t = -4.39, 
p <.00 1), getting a complete picture of weather (t = - 
3.14, p < .01), and planning vertical maneuvers using 
RAT (t = -3.52, p < .01). Perspective 3D and 2ViD 
presentations received similar ratings in terms of their 
support for assessing the spread of weather and 
planning lateral maneuvers. 

In Experiment 2 pilots were also asked to 
indicate the extent of forecast range that they 
consider could be used effectively (Figure 7). Again 
almost all of the pilots considered forecasts of up to 
an hour in range to be useful. 

Presentation Mode Usage (Perspective 3D 
Condition Only) 

In the perspective 3D condition pilots could 
freely change the viewpoint of CSD and set its view 
to 2D if desired. Although the majority of pilots 
preferred perspective 3D over 2 V 2 D, almost all of 
them set the presentation mode to 2D some portion of 
the time. While the proportion of time spent in 2ViT> 
view varied widely among pilots, on average they 
spent about 40% of the time viewing weather in 
2ViT>. The time portion of 2ViT> viewing increased 
even more, to around 47%, when pilots were actively 
using the RAT to modify trajectories. 

In summary, despite a lack of objective 
performance benefits, pilots expressed a preference 
for perspective 3D presentation of weather. It should 


Table 4. Experiment 2 Subjective Evaluation Results 



Weather assessment 

Route planning 

Presentation 

Condition 

Height of 
Wx 

Spread of 
Wx 

Penetration 
of planned 
trajectory 

Penetration 
of trajectory 
being 
planned 

Complete 
picture of 
Wx 

Lateral 

maneuvers 

Vertical 

maneuvers 

2V2D 

3.00 

4.17 

3.25 

3.18 

3.62 

4.25 

3.25 

Perspective 

3D 

4.18 

4.33 

4.18 

4.12 

4.31 

4.18 

4.18 






Figure 7. Experiment 2 Subjective Effective 
Forecast Range 

be noted that the perspective 3D presentation 
condition used in the present experiment also 
included 2 l AD presentation. Therefore, it is not 
surprising that pilots might prefer a presentation 
condition that provides more ways to view weather. 
However, giving this additional information to the 
pilots did not appear to hurt performance, nor did it 
lengthen the amount of time needed to complete the 
task. All in all the present results suggest that a 
presentation mode combining 2 l AD and perspective 
3D has the potential for providing a more 
comprehensive weather picture for the pilots without 
negative impact on performance 


General Discussion 

The results of two part-task experiments showed 
that pilots could take advantage of the additional 
forecast information while performing en route 
weather avoidance but more research will be needed 
to tailor the presentation mode to pilots’ use. In 
particular, one issue that arose unexpectedly from the 
course of the study concerns the scale of weather 
displays. In the experiments pilots were able to 
visualize the complete 2-hour segment of the 
trajectory to be flown and the surrounding weather, 
as far as 1000 NM ahead. Because pilots are used to 
seeing only about 300 NM ahead, the scale of our 
display may have led them to misjudge distances. 
Because of the scale issue, it is probably important 
that in the NextGen Trajectory-Based Operation 


(TBO) environment, pilots have access to features 
like panning and zooming so that they have a better 
indication of when their routes will take them within 
some unacceptable distance to the storms. The ability 
to pan over to weather impacted portions of their 
trajectory will allow pilots to examine them at a 
much lower scale. 
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